Performance Analysis of Publish/Subscribe Systems

The Desktop Grid offers solutions to overcome several challenges and to answer increasingly needs of scientific computing. Its technology consists mainly in exploiting resources, geographically dispersed, to treat complex applications needing big power of calculation and/or important storage capacity. However, as resources number increases, the need for scalability, self-organisation, dynamic reconfigurations, decentralisation and performance becomes more and more essential. Since such properties are exhibited by P2P systems, the convergence of grid computing and P2P computing seems natural. In this context, this paper evaluates the scalability and performance of P2P tools for discovering and registering services. Three protocols are used for this purpose: Bonjour, Avahi and Free-Pastry. We have studied the behaviour of theses protocols related to two criteria: the elapsed time for registrations services and the needed time to discover new services. Our aim is to analyse these results in order to choose the best protocol we can use in order to create a decentralised middleware for desktop grid

Parallelization of Littlewood-Richardson Coefficients Computation and its Integration into the BonjourGrid Meta-Desktop Grid Middleware

International audienceThe aim of this paper is to show how to parallelize a compute intensive application in mathematics (Group Theory) for an institutional Desktop Grid platform coordinated by a meta-grid middleware named BonjourGrid. The paper is twofold: first of all, it shows how to parallelize a sequential program for a multicore CPU which participates in the computation and second it demonstrates the effort for launching multiple instances of the solutions for the mathematical problem with the BonjourGrid middleware. BonjourGrid is a fully decentralized Desktop Grid middleware. The main results of the paper are: a) an efficient multi-threaded version of a sequential program to compute Littlewood- Richardson coefficients, namely the Multi-LR program and b) a proof of concept, centered around the user needs, for the BonjourGrid middleware dedicated to coordinate multiple instances of programsfor Desktop Grids and with the help of Multi-LR. In this paper, the scientific work consists in starting from a model for the solution of a compute intensive problem in mathematics, to incorporate the concrete model into a middleware and running it on commodity PCs platform managed by an innovative meta Desktop Grid middleware

D 3 -MapReduce: Towards MapReduce for Distributed and Dynamic Data Sets

International audienceSince its introduction in 2004 by Google, MapRe-duce has become the programming model of choice for processing large data sets. Although MapReduce was originally developed for use by web enterprises in large data-centers, this technique has gained a lot of attention from the scientific community for its applicability in large parallel data analysis (including geographic, high energy physics, genomics, etc.). So far MapReduce has been mostly designed for batch processing of bulk data. The ambition of D 3-MapReduce is to extend the MapReduce programming model and propose efficient implementation of this model to: i) cope with distributed data sets, i.e. that span over multiple distributed infrastructures or stored on network of loosely connected devices; ii) cope with dynamic data sets, i.e. which dynamically change over time or can be either incomplete or partially available. In this paper, we draw the path towards this ambitious goal. Our approach leverages Data Life Cycle as a key concept to provide MapReduce for distributed and dynamic data sets on heterogeneous and distributed infrastructures. We first report on our attempts at implementing the MapReduce programming model for Hybrid Distributed Computing Infrastructures (Hybrid DCIs). We present the architecture of the prototype based on BitDew, a middleware for large scale data management, and Active Data, a programming model for data life cycle management. Second, we outline the challenges in term of methodology and present our approaches based on simulation and emulation on the Grid'5000 experimental testbed. We conduct performance evaluations and compare our prototype with Hadoop, the industry reference MapReduce implementation. We present our work in progress on dynamic data sets that has lead us to implement an incremental MapReduce framework. Finally, we discuss our achievements and outline the challenges that remain to be addressed before obtaining a complete D 3-MapReduce environment

Approches de décentralisation de la gestion des ressources dans les Grilles de PC

Les grilles de PC offrent une puissance de calcul et de stockage, avec un coût économique faible, grâce à la mutualisation des ressources sous exploitées. Leurs performances sont fortement liées à la participation volontaire des utilisateurs qui mettent leurs machines disponibles lorsqu'elles sont inexploitées. Plusieurs critères, tels que le nombre et la volatilité des ressources, la sécurité et la faible performance de communications, font que la gestion des ressources des grilles de PC soit un grand défit à réaliser. Nous nous intéressons à concevoir des approches décentralisées de gestion des ressources de grilles. Nous avons commencé par réaliser une étude de performance des protocoles de découverte de ressources pouvant servir comme base des systèmes de découverte de ressources. Par la suite, nous avons conçu et développé un premier système de grilles de PC, appelé BonjourGrid, ayant pour objectif de déployer des systèmes de calcul existants tout en décentralisant la gestion des ressources. Avec BonjourGrid, il est possible de construire un système de calcul à la demande en précisant le type d'ordonnanceur, le nombre de ressources, etc. Ensuite, nous avons proposé un deuxième système de gestion de ressources décentralisé et tolérant aux fautes pour les grilles de PC, appelé PastryGrid. Il est capable de gérer les applications distribuées avec précédentes entre les tâches. Il est basé sur une table de hachage distribuée utilisant les réseaux structurés. Le principe de PastryGrid est de créer pour chaque application son environnement d'exécution d'une manière dynamique sans recours à aucun élément central. Son approche de coordination est collaborative et distribuée.Desktop Grids provide computing and storage power, with a low economic cost, through the federation of free resources. Their performance relies strongly on the voluntary participation of users who make their machines available when these are idle. Several criteria such as number and volatility of resources, security and the low performance of communications, make resources management, in desktop grids, a great challenge. We are interested in designing decentralized approaches for resources management in desktop grids. We began with a performance study of resources discovery protocols. Then, we designed and developed our first desktop grid system, called BonjourGrid, aiming to deploy existing computing systems while decentralizing resources management. With BonjourGrid, it is possible to build, on demand, a computing system specifying the scheduler type, the resources number, etc. Then, we proposed a second decentralized and fault tolerant desktop grid system, called PastryGrid. It is able to manage distributed applications with precedences between tasks. It is based on a distributed hash table using structured networks. Its principle is to create, for each application, a runtime environment dynamically and without recourse to any central element. Its coordination approach is collaborative and decentralized.PARIS13-BU Sciences (930792102) / SudocSudocFranceF

From desktop grid to cloud computing based on BonjourGrid middleware

n/

BonjourGrid as a Decentralised Job Scheduler

International audienceInstitutional desktop grid systems are attractive for running distributed applications with significant computational requirements. While the rapid increasing number of users and applications running on such systems does demonstrate the potential of desktop grid and institutional desktop grid, current implementations follow the old-fashioned master-worker paradigm. Obviously, vulnerability to failures and permanent administrative monitoring are the disadvantages of client-server architectures. To bypass this, we proposed a novel system, called BonjourGrid, able to orchestrate multiple instances of institutional desktop grid middlewares, able to remove the risk of single-source bottleneck and failure, and able to guaranty the continuity of services in a distributed manner. In this paper, we use BonjourGrid protocol, which is based on publish/subscribe paradigm, to show how to adapt it to fulfill all the requirements of a decentralised job scheduler. An evaluation proves that BonjourGrid is able to manage more than 100 applications instanciated in a concurrent way on an institutional desktop grid. Analysing the execution of 100 applications with 2110 tasks during 3 hours demonstrates the potential of BonjourGrid concept and shows that, comparing to a classical desktop grid with one central master, Bonjourgrid gives an acceptable overhead that can be explained

PastryGrid: decentralisation of the execution of distributed applications in desktop grid

International audienceThis paper proposes a decentralised system for managing Desktop Grid (DG). The idea is to bypass the main drawback of existing systems putting all the control on a single master that can fails. Here, each node can play alternatively the role of client or server. Our main contribution is to design the PastryGrid protocol (based on Pastry) for DG in order to decentralise the execution of a distributed application with precedence between tasks. Comparing to a centralised system, we evaluate our approach over 205 machines executing 2500 tasks. The results show that our decentralised system runs better than the same system configured as a master/slave because it gives less overhead

Multithreading of Kostka Numbers Computation for the BonjourGrid Meta-desktop Grid Middleware

International audienceThe aim of this paper is to show how to multithread a compute intensive application in mathematics (Group Theory) for an institutional Desktop Grid platform coordinated by a meta-grid middleware named BonjourGrid which is a fully decentralized Desktop Grid middleware. The paper is twofold: first of all, it shows how to multithread a sequential program for a multicore CPU which participates in the computation of some parameters and second it demonstrates the effort for coordinating multiple instances of the BonjourGrid middleware. The main results of the paper are: a) we develop an efficient multi-threaded version of a sequential program to compute Kostka numbers, namely the Multi-kostka program and b) a proof of concept is given, centered on user needs, for the incorporation into the BonjourGrid middleware of Multi-kostka program

Architecture d'une grille de calcul Tunisienne

http://www.esstt.rnu.tn/jtea06/jtea06.htm